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The use of transversal filters for automatic equalization has made 
possible high-speed data communication over voiceband telephone 
channels. Recently much attention has been focused on the possible 
use of the high-speed data sets in private line multiparty polling 
systems. However, for such applications, it is necessary to reduce 
the start-up time of the present automatic equalizer drastically. This 
paper examines the start-up time (settling time) of the transversal 
filter equalizer for two important classes of data communication 
systems: Class IV partial-response systems and single-sideband 
Nyquist systems. (The latter represents the limiting case of vestigial- 
sideband systems with small roll-off bandwidth.) It is shown that in 
single-sideband Nyquist systems the input signals to the gain controls 
of the transversal equalizer may be nearly orthonormal. Consequently 
the equalizers may have a short settling time. It is also shown that the 
equalizer settling time is much longer in Class IV partial-response 
syste7ns, because such systems use controlled intersymbol interference 
and the input signals to the gain controls are highly correlated. 

The possibility of reducing the settling time of the automatic 
equalizers is examined. A new equalizer structure is developed based 
on the following principles: (i) Equalizer settling time can be mini- 
mized by making the input signals to the gain controls orthonormal, 
and (ii) Such a minimization does not change the noise power, the 
mean-square equalization error, the convexity of the gain control 
adjustment, and the feedback control loops in the equalizer. These 
principles are general in that they apply regardless of the type of 
modulation — single- , vestigial- , or double-sideband (SSB, VSB, or 
DSB) — or the signaling scheme (Nyquist or partial-response) . Appli- 
cation of these principles to Class IV partial-response systems is con- 
sidered. For private line systems and systems where amplitude 
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distortions in the communication channels are not severe (delay 
distortions can be arbitrary), the new equalizer can be implemented by 
simply adding a prefixed weighting matrix to the conventional trans- 
versal equalizer. Analysis and computer simulation show that the 
use of such a new equalizer can residt in a significant reduction in the 
system's start-up time. 

I. INTRODUCTION 

In order to meet the needs of the rapidly growing computer and 
data processing industries, a number of high-speed data sets have been 
developed in recent years for voiceband telephone channels. Most of 
these data sets use transversal filters 1 for precise automatic equaliza- 
tion. The transversal equalizer consists of a tapped delay line with 
variable tap gains. During a start-up period prior to data transmis- 
sion, the tap gains are adjusted automatically to minimize the peak 
distortion 1 or the mean-square error 2 - 3 - 4 of the received pulses. The 
time required to adjust the tap gains to nearly their optimum settings 
is usually called the settling time of the equalizer. Most automatic 
equalizers have settling times of a few seconds. The start-up time of 
the system can be longer because there are usually other operations to 
be performed in the start-up period (operations such as synchroniza- 
tion, carrier recovery, and so forth) . 

Recently, much attention has been focused on the possible use of 
the high-speed data sets in private-line multiparty polling systems 
(such as airline reservation systems, on-line banking systems, and so 
forth). Such systems are generally real-time information retrieval 
systems where the inquiry and response are short (the message lengths 
are usually less than 1000 bits 5 ) . With a 4800 b/s data set, it takes 
only 0.208 second to transmit a message of 1000 bits. Consequently 
the actual transmission time can be much less than the start-up time 
of the system (which can be five seconds). In order to allow addi- 
tional stations to be served or to reduce the response time of the 
system (response time is important in real-time systems), it is 
necessary 5 to reduce the start-up time of the high-speed data sets 
drastically. This means that the settling time of the automatic 
equalizer must be reduced drastically (for example, from a few seconds 
to tens of milliseconds) . 

Such a drastic reduction in the start-up time of the high-speed data 
sets raises a number of theoretical questions. For example, it is no 
longer sufficient to just prove the convergence of the equalizer adjust- 
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ment; it must be shown that the convergence is sufficiently fast to 
meet the requirement. It becomes necessary to examine the dependence 
of equalizer settling time on modulation and signaling schemes. Instead 
of considering synchronization, carrier recovery, and automatic equali- 
zation separately, one has to consider the mutual dependence of these 
adjustments and the possibility of minimizing the overall adjustment 
time. The start-up time requirement also provides a strong motivation 
to search for a new equalizer that has a settling time shorter than 
that of the conventional transversal equalizer. Answers to some of 
these problems are presented in this paper. Because the paper is 
lengthy, the contents of each section are outlined here; the results 
are summarized in Section VIII (the reader may read Section VIII 
first) . 

Section II includes a description of the mathematical model and 
reviews some of the fundamental works on automatic equalization 
(particularly those by Lucky and Gersho). Section III examines 
the equalizer settling time for two important classes of digital com- 
munication systems : the Class IV partial-response system and the SSB 
Nyquist system. (The latter represents the limiting case of VSB 
systems with small roll-off bandwidth.) Surprisingly, the results show 
that the equalizer settling times of these two systems can be very 
different. The reason for this difference (eigenvalue spread) is ex- 
plained so that the method of analysis can be extended to other sys- 
tems. Sections IV and V consider a general data communication 
system and develop a new equalizer structure for fast start-up pur- 
pose. These sections stress the underlying principle (condition of 
orthogonality) and analyze the various properties of the new equalizer 
(including convergence rate of equalizer adjustment, residual noise 
power, minimum mean-square error, and convexity of the adjustment). 
In Sections VI and VII, application of the new equalizer to the Class IV 
partial-response system is considered. Most importantly, it is shown 
that the new equalizer can be implemented by simply adding a pre- 
fixed weighting matrix to a conventional transversal equalizer. The 
related analytical studies and computer simulation are described; 
fast convergence of the equalizer adjustments is demonstrated. Sec- 
tion VIII is a summary of the results. 

II. REVIEW OF FUNDAMENTALS 

An amplitude modulation data communication system utilizing a 
transversal equalizer is depicted in Fig. 1. The equalizer consists of 
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Fig. 1 — Block diagram of an amplitude modulation data communication sys- 
tem with a conventional transversal equalizer. 

a delay line tapped at T-second intervals, where T is the signaling 
interval of the system. The ith. tap, i = 1 to N, is connected through a 
variable gain control c { to a summing bus. During data transmission, 
the transmitter transmits the information digits sequentially at time- 
instants t = • ■ • , h - T, t r , t x + T, t! + 2T, • • • . The equalizer 
output is sampled sequentially at time-instants t = • • • , t 2 — T, 
1 2 , £2 + T, t-2 + 2T, • • • , and the time-samples are used to recover 
the information digits. To simplify the notations, we shall shift the 
origin of the time-axis to make t 2 = 0. 

When an impulse 8(£ — £1) is applied at the transmitter input, the 
equalizer input and output are, respectively, x(t) and y(t). Since we 
consider only linear systems, y{t) is the overall impulse response. The 
desired overall impulse response of the system is d(t). In this study, 
we adopt the familiar mean-square error criterion 3 ' 4 - and adjust the 
gain controls of the equalizer to minimize the mean-square error 
between y(t) and d{t) at the receiver sampling instants t = ••• , 
— T, 0, T, 2T, • • • . The mean-square error can be written as 



e = E WT) - d(iT)]\ 

t =— 00 

It can be seen from Fig. 1 that 

y(t) = T,c k x[t- (k- 1)71 



(1) 



(2) 



For the sake of simplicity, we shall use the abbreviations y t = y (iT) , 
di = d{iT), and x t = x{iT). It can be seen from (2) that (1) can be 
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written in the following matrix form 

e = c'Ac - 2c'v + J) <fi , 

i_— 00 

where 

r Ci 

c, 
c = 



_C.v_ 

a,, a, 2 ••• auv 
a 2 i <ia2 ■ • • o, 2 .\ 



A = 



_a j \i fl.v2 ■ ■ ■ o,\\_ 

«.y = £ .i-/_ 1 + 1 .i; / _, + , , all i, j, 



v = 



v k 



= 2 •t'.-A+i^. , all k. 
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(•3) 



(4) 



(5) 



(0) 



(7) 



(8) 



It can be shown that A is positive definite. 

Let Be/dCi be the partial derivative of e with respect to c ( , i = 1 to 
N, and let de/dc represent an .V x 1 column vector whose ith element 
is de/dc,-; i.e., 

dCi 



de 
dc 



de_ 
dc 2 
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From (1), we obtain 

£ = 2Ac - 2v. (9) 

The measured de/dc in actual data sets will deviate 4 from that in (9) 
due to noise in the receiver. However this deviation is negligible in 
high signal-to-noise ratio systems. For example, it has been found 
(Section 7.3) that at a 30-dB signal-to-noise ratio this deviation has 
only a minor effect on the settling time of the equalizer. In this paper 
we consider high signal-to-noise ratio systems and neglect such a 
deviation. 

The optimum value of c that minimizes the mean-square error e will 
be called c^t . It is clear that c minimizes c if and only if de/dc = 0; 
therefore, from (10) the optimum c is 

c opt = A-v. (10) 

The difference between c and c opt is denoted by e: i.e., 

e = c - A-V. (11) 

Let € min be the minimum value of e when c = c^t . From (3) and (10) 

e mi „ =Zd!- v'A-'v. (12) 

1 =-00 

Now consider the adjustment of the equalizer. As is well known, 3 - 7 
the equalizer can be adjusted in the training period prior to data 
transmission by transmitting either a succession of isolated test pulses 
or a sequence of Pseudo-random numbers. Since these two methods 
differ considerably, we shall consider only the first method (isolated 
test pulses) in this paper. 

In the training period, isolated impulses are applied to the trans- 
mitter input. For instance, S(t - h) in Fig. 1 may be the fcth such 
impulse. The transmission of S(t - h) produces the test pulse x(t) at 
the equalizer input. From x{t) the partial derivatives de/dci , i = 1 
to N, are computed 3,4 ' 6 and the gain control c* is changed by an amount 
proportional to de/dc* . This process is then repeated for the next 
test pulse. 

The adjustment made after the fcth test pulse, k — 1, 2, 3, • • • , will 
be referred to as the /cth adjustment. The initial values of c, e, and e 
(i.e., their values prior to the first adjustment) will be denoted, 
respectively, by c , e , and eo . The values of c, e, and « after the 
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fcth adjustment are denoted, respectively, by c k , e k , and c fc . From 
(11) 

e t = c* - A-v k = 0, 1, 2 • • • . (13) 

It can be easily shown from (3), (12), and (13) that 

H = e min + e£Ae*, fc -0,1,2, ... . (14) 

The fcth adjustment is made according to the equation 



! * ~ Ca - 2 a *Ucl' 



c > = c ^-2 a iTc] k > (15) 

where « fc , k = 1, 2, 3, • • • , are suitably chosen constants. 4 

The subscript k of de/dc indicates that the dt/dc is computed from 
the fcth test pulse. It can be shown from (9), (15), and (13) that 

e, = (I - a,A)e,_, , k = 1, 2, 3, • • • . (16) 

We now proceed to study the convergence of the mean-square error 
for several data communication systems. 

in. A STUDY OF CONVERGENCE FOR PARTIAL-RESPONSE AND NYQUIST 

SYSTEMS 

We have detined e k as the value of e after the fcth adjustment. Note 
from (14) that e* consists of two terms. The first term e min is the ir- 
reducible value of e. Only the second term e£Ae t depends on c and the 
adjustments. Thus, we shall study the convergence of the second 
term in this section. 

It can be seen from (6) that A is a symmetric matrix. Let the eigen- 
values of A be denoted, in the order of increasing magnitude, by X; , 
i = 1 to Af, so that 

X. ^ X 2 ^ • • • ;g \„ , (17) 

and let \ii , i = 1 to N, be a set of orthonormal eigenvectors of A (Ui is 
the eigenvector corresponding to A<). It is well known that A can be 
represented in the form 

A = QDQ', (18) 

where D is an iV X .V diagonal matrix whose ith diagonal element is 
At , and Q is an N X N matrix whose ith column is the eigenvector U* . 
It is also well known that Q is an orthogonal matrix; i.e., 

Q' - Q-\ (19) 
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From (18) and (19), 

I - a k A = Q[I - «*D]Q'. (20) 

By repeated application of (16), one obtains 

e, - (I - a*A)(I - « t -iA) • • • (I - a 2 A)(I - a,A)e 

= fl(I-a n A)e fl , *- 1,2,3, ■•• . (21) 

n=l 

Substituting (20) into (21) and noting that Q'Q = I gives 

e* - q[E[ (I " « n r>)jQ'e , k m 1, 2, 3, • • • . (22) 
From (22) and (18), 

e' k Ae k = e^[ll (I - «« D )] D [lI (I " ^)]Q'^ , 

k = 1,2, 3, ••• . (23) 

Using the properties of D and Q described one can carry out the 
matrix multiplications in (23) and obtain 

e^Ae, = £ *«<*). k= 1,2, 3, ••• . (24) 

where 

Uk) = (e5a,-) 8 X.-[lI(l -«nX,) 2 ] , t-1 to N. (25) 
In a similar manner, we find that 

e£Ae = E^(0), (26) 

i-l 

where 

5,(0) = (fifrdX, i = ltoN. (27) 

Since A is positive definite, A, > for all i. Thus, fc(fe) ^ for all %■ 
Consequently, e£Ae* converges to zero if and only if £,(/c), i = 1 to N, 
all converge to zero (see (24)). For this reason, £,(fc) will be called the 
?'th error component. If the error components all converge rapidly 
to zero as k increases, e' k Ae k converges rapidly to zero. 

It is clear from (25) that £,(fc) converges to zero if and only if the 
factor 
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[n <!-«*«>•] 



converges to zero. Thus, the convergence of &(fc) to £»(&) depend 
only on two sets of parameters: ai , ■ • ■ , a* and Ai , • • • ,X N . The first 
set of parameters corresponds to the magnitudes of the gain-control 
adjustments [As can he seen from (15), a k determines the magnitude 
of the feth adjustment.]. In the following subsections, we show that the 
second set of parameters, Ai , • • • , A. v , depend on the modulation 
scheme and channel characteristics. In some systems, Ai , • • • , A, v 
differ only slightly in value. Consequently a k can be selected such 
that each adjustment reduces each of the error components by a large 
factor (such as 100). However, this is not possible in some other 
systems. 

3.1 Class IV Partial-Response System 

There are several classes of partial-response systems. 8 We shall 
consider the most important one: Class IV partial response sys- 
tem. 9 - 10,1112,13 The results can be easily extended to other classes. 

As depicted in Fig. 1, the transfer functions of the transmitting 
filter, transmission medium, and receiving filter are, respectively, Ft (/) , 
/*\>(/), and F a (f). The amplitude and phase characteristics of F<(/) 
will be denoted, respectively, by |F,-(/)| and /?,(/), i.e., 

F<(1) = | F,(f) | e«" lf) , i = 1, 2, 3. (28) 

N ote t hat in this paper J will be used to denote the imaginary number 
V— 1 is used as an index). 

In a Class IV partial-response system, the transmitting and the 
receiving filters are band-limited; that is, 

I FxfflF.fl) | = 0, 
when 

I / I £ /i and | / | fc /, , (29) 

where /i and f- 2 are, respectively, the lower and the upper cutoff 
frequencies. The demodulating carrier frequency, f r , is usually equal to 
ft . This implies that the system is SSB. According to the previous 
definition, when an impulse 8{t — t^) is applied at the transmitter 
input, the equalizer input is x(t). Let X(f) denote the Fourier trans- 
form of x(t). It can be shown from (28), (29), and the demodulation 
process that 



1978 THE BELL SYSTEM TECHNICAL JOURNAL, JULY-AUGUST 1971 

X(1) = I \F t (j - f e )F a (j - j.)F*(f - f.)\ 

•exp ( Jflfcfl - f e ) + M ~ U + M ~ U) ~ Mf ~ 1c)k + d]} f 

= t \Fx(j + fe)F 2 (j + j t )F a (f + f e )\ 

■ exp { J[/3,(/ + f„) + &(/ + /,) + 0,0 + /,) - 2tt(/ + U)U ~B]\, 

-{U - A) £ f £ 
= 0, all other frequencies. (30) 

Now we can determine the A matrix. From (6) , the elements of A 
are 

00 

«.;• = £ %1-i+lXl-i+l ■ ( 6 ) 

; = -« 

In a Class IV partial-response system the signaling interval is 

T = W=T) (31) 

It is clear from (30) that x(t) is band-limited from to (/ 2 - /i) Hz. 
Thus, the time-samples x(kT), k = • • • , 0, 1, 2, • • • , are taken at 
the Nyquist rate. Therefore, according to the sampling theorem 

f" x[t - iT + T]x[t - jT + T]dt = ^ 7T E x t - i+1 x t . i+1 . 

J-oo ^' 2 ~ i 1 ' •— « (32) 

Comparing (6) and (32) shows that 

*, = 2(/ 2 - /,) I" x[t - iT + T]x[t - jT + T) dt. (33) 

J-oo 

From Parseval's theorem and (30), one can rewrite (33) as 

C„ = (/a - /■) f 3 ' t C0S ^ " j)T] 
Jo 

[\Ftf - j e )F a (f - j e )F a (j - f e )\] 2 df. (34) 
It can be seen from (34) that ay is independent of the following 
parameters: demodulating carrier phase 6, system timing h , phase 
characteristics of the transmitting and receiving niters, and phase 
characteristic of the transmission medium. To evaluate ay , we need to 
specify only the amplitude characteristics \F x {j — f c )\, \F 2 (f — fc)\, 
and \F 3 (f - / c )|. In a Class IV partial-response system, the trans- 
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mitting and receiving filters are designed such that 

7-/, 



1979 



\FM «A| =sinx 



U£f£h 



(35) 



J. ~ /.J ' 
= sin7r[ ~ / J / |' ] , -U*f±-1 

Substituting (35) into (34) gives 

Ja 

• [(sin r f — L) \F 2 (f - f e )\j dj. (36) 

We first consider the case where the transmission medium has a 
constant amplitude characteristic 

WM - 1 (37) 

in the pass band /i ^ / ^ jo. Substituting (37) into (36) and evaluat- 
ing the integral, we obtain 

0. - uy 



a,, = 



2 



i — j = — 2 or 2 



i - ; = 



= 0, all other t - j. (38) 

The constant term (/ 2 — /i) 8 /2 in a,, may be dropped (this corresponds 
to introducing a gain of V§/(/ 2 — /0 in the channel). Then (38) becomes 



a ti = -h i~ j = -2 or 2 
= 1, i - j = 

= 0, all other t — j. 

Thus, the matrix A is of the following form 

1 -$ ••• 

1 -\ ••• 

■h 1 ••■ 



(39) 



A = 



0-4 



1 











(40) 
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In particular the main diagonal elements are 1, the second off- 
diagonal elements are -.1/2, and the other elements are all (notice 
that A belongs to a class of matrixes known as the "Toeplitz" 14 ) . We 
have defined \ t to \ N as the eigenvalues of A, and u x to u N as a set 
of orthonormal eigenvectors of A. These eigenvalues and eigenvectors 
are determined in Appendix A. Since transversal equalizers usually 
have an odd number of taps, we shall assume that JV is odd in the 
following discussion. From Appendix A, the N eigenvalues of A are 



1 — cos 



far 



N + 1 



+ 1 



*-l,2, 



N + 1 



(41) 



and 



1 — cos 



fcir 



N - 1 



+ 1 



ft- 1,2, 



N - 1 



(42) 



Thus, the eigenvalues of A are points on the curve 1 - cos 0, < 6 
< n (see Fig. 2). The minimum eigenvalue 



1 — cos 



N + 1 



+ 1 




Fig. 2— Distribution of the eigenvalues of A(\mi» and X m ai shown are for 
N = 17). 
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and the maximum eigenvalue 

N + 1 
1 — cos 



N + l + 1 



2 

are also shown in Fig. 2. Other eigenvalues lie between these two 
points. It can be seen from this figure that, for typical values of N 
(13, 17, etc.), the minimum eigenvalue is very close to and the 
maximum eigenvalue is veiy close to 2. For example, consider a trans- 
versal equalizer with 17 taps (N = 17). From (41) and (42), the 17 
eigenvalues of A are 0.0489, 0.0603, 0.1910, 0.2340, 0.4122, 0.5, 0.6910, 
0.8263, 1, 1.174, 1.309, 1.5, 1.588, 1.766, 1.809, 1.940, and 1.951. 

When the amplitude characteristic of the transmission medium is 
not exactly a constant in the passband / x ^ / =§ / 2 , the calculations 
from (38) to (42) will change slightly. Consequently the eigenvalues 
will differ slightly from those above. 

Having determined the distribution of the eigenvalues, we now 
consider the convergence of the error components &(fc), i = 1 to N. 
Prior to the fcth adjustment, the ith error component is 

Uk - 1) = (eX) 2 X,[lI (1 - a»X,) 8 ] • (43) 

After the fcth adjustment, the ith error component is 

€<(*) - (e£u,) 2 A<[n (1 -a„X,.) 2 ]. 
Substituting (43) into the above equation gives 

£,(/>•) = (1 - «,X,r £,(/< - 1), k = 1, 2, 3, • • • . (44) 

It is easily seen from (44) that the fcth adjustment will reduce each 
of the error components by a large factor if 

(1 - a k \,) 2 « 1, for alii 
or if 

a k ^~ , for all i. (45) 

Unfortunately, a k cannot satisfy (45) because Ai , ■ ■ • , X s are very 
different in value. Therefore, the fcth adjustment cannot reduce each 
of the error-components by a large factor. To illustrate this quantita- 
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tively, consider again the 17-tap transversal equalizer (A x = 0.0489 
and A n = 1.951). In order to reduce the first error-component &(fc), 
we set (1 - a fc Ai) 2 « 1 or « fc = 1/A X = 20.4. However, this a k will 
cause a large increase in some other error-components. For example, 
the 17th error-component will increase about 1505 times because 
(1 - a k \ xl ) 2 = (1 - 20.4 X 1.951) 2 = 1505. On the other hand, if we 
wish to reduce the 17th error-component, we should set (1 — a k kn) 2 
«. 1, or a k = I/A17 = 0.512. However, this a k is too small to reduce 
some other error-components rapidly. For instance, the first error- 
component will reduce only five percent because (1 — a fc Ai) 2 = 
(1 - 0.512 X 0.0489) 2 = 0.95. These examples clearly demonstrate 
that the fcth adjustment cannot reduce each of the error-components 
by a large factor. 
"We now summarize the results in this subsection: 

(i) In a Class IV partial-response system, the eigenvalues Xi to 
X, v of the A matrix depend only on the amplitude characteristic 
of the channel, but not on carrier phase, system timing, and 
phase characteristic of the channel. 

(«) These eigenvalues are very different in value. For typical values 
of N (13, 17, etc.), the minimum eigenvalue X, is very close 
to 0, while the maximum eigenvalue X^ is very close to 2. 

(iii) Because of the large differences in the eigenvalues, each adjust- 
ment of the gain controls cannot reduce each of the error- 
components by a large factor. 

3.2 SSB Nyquist Systems 

In this section, we study SSB data communication systems which 
transmit at the Nyquist rate with sin x/x pulses (hereafter referred 
to as SSB Nyquist systems) . Such a system has the same configuration 
as the Class IV partial-response system except that its transmitting 
and receiving filters have constant amplitude characteristics in the 
passband. The main advantage of an SSB Nyquist system is that it 
transmits at the maximum possible bauds (Nyquist rate) without noise 
penalty. Saltzberg 15 has shown that this signaling method is not as 
sensitive to timing error as is commonly believed. In fact we show 
in this section that this scheme exhibits fast start-up advantages. 

The transmitting and receiving filters are specified by 

\FM - 1, U ^ l/l ^ U 

= 0, other/. (46) 
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and 

\FM = 1, ft £ l/l ^ / 2 

= 0, other /. (47) 

The signaling interval T is again given by (31), and ay by (34). Sub- 
stituting (46) and (47) into (34) , we have 

«./ = (U -/•)/" ' [cos 2*f(t - j)T] \F 2 (f - f e )\ 2 df. (48) 

Jo 

Note again that ay (and hence the eigenvalues Ai to k N ) depends only 
on the amplitude characteristics of the transmission medium and the 
filters. When the transmission medium has a constant amplitude 
characteristic 

I F 2 (j) | = 1 
in the passband /i ^ / ^ / 2 , (48) yields 

a u = (f 2 - /,) 2 , i - j = 

= 0, i - j ^ 0. (49) 

Neglecting the constant (/ 2 — /i) 2 above, we see that A is simply the 
identifying matrix and the eigenvalues of A are 

A, = 1, i - 1 to N. (50) 

Now consider the convergence of the error components &(&), i = 1 to 
N. Prior to the first adjustment, the ith error component is 

g,(0) = (eX) 2 X, • (51) 

After the first adjustment, the ith error component is 

£,(1) = (e,' )Ul ) 2 A,(l - ai X,) 2 . (52) 

Substituting (50) and (51) into (52) yields 

£,(1) = (1 - ai )%(0). (53) 

If we set «i = 1, |i(l) =0 for all i. In other words, when ai = 1, the 
first adjustment reduces all the error-components to zero. Conse- 
quently the mean-square error e is reduced to its irreducible value 
Cni,, after only one adjustment. This fastest possible convergence is 
obtained regardless of the initial equalizer settings, the carrier phase 
and system timing, the phase characteristic of the transmission 
medium, and the phase characteristics of the transmitting and receiv- 
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ing filters (because the eigenvalues Ai to Ay are independent of all 
these parameters). 

From (48), a u can be computed for any given \F 2 (f)\. When 
\F 2 {f)\ is not exactly a constant in the passband, A will not be an 
identity matrix and the eigenvalues will not be all equal. However, 
fast convergence is obtainable as long as the differences between the 
eigenvalues are small. For example, consider the case where the 
minimum eigenvalue is 0.9 and the maximum eigenvalue is 1.1. If we 
use«i = 1, the maximum value of (1 - «iA/) 2 will be (1 -0.9) 2 = 0.01. 
Consequently the first adjustment will reduce each of the error com- 
ponents by at least a factor of 100 (as will each following adjustment). 
The equalizer adjustment therefore will be completed after only a 
few adjustments. 

IV. A NEW EQUALIZER STRUCTURE 

In the preceding sections, we have clearly shown that, when the 
eigenvalues of A have close magnitudes, each adjustment of the 
equalizer reduces each of the error-components by a large factor (such 
as 100) . We have also shown that such a fast convergence is not pos- 
sible when the eigenvalues are very different in magnitude (such as in 
the case of a Class IV partial-response system) . These results suggest 
that in order to improve the convergence rate we should attempt to 
reduce the differences between the eigenvalues. 

Now we ask: What causes the eigenvalues to be different? Is it 
possible to reduce such differences (and hence increase the con- 
vergence rate) by changing the equalizer structure? Does the use oj 
the new equalizer structure alter the system's performance otherwise? 
We shall consider the first two questions in this section, and derive a 
new equalizer structure. The last question will be considered in the 
next section. Application of the results to a Class IV partial-response 
system and the various related problems will be studied in Sections VI 
and VII. 

It may appear that the new equalizer structure derived in this 
section requires complicated mathematical operations (computation 
of eigenvalues and eigenvectors of a matrix). However, it will be 
shown in Section VI that such mathematical operations can be com- 
pletely eliminated for the systems of interest here. 

Now consider the first question: What causes the eigenvalues to 
be different? From the definition in (6) 
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an - E *[(* - » + l)r]a;[(Z - j + i)rj. 

Since the input to the ith gain control is x[t — (i — l)T], x[ (I — i + 
1)T] are time-samples of the input to the ith gain control, while 
x [il ~ J + VT] are time-samples of the input to the ;'th gain control. 
Therefore, a. - is simply the cross-correlation between the inputs to the 
ith and the ;th gain control, and A has the interpretation of a corre- 
lation matrix. Therefore, we might state that it is the correlations 
between the inputs to the gain controls that result in the differences 
between the eigenvalues. When the inputs to the gain controls are 
orthonormal, A is an identity matrix and the eigenvalues are equal. 
When the inputs to the gain controls are correlated, the eigenvalues 
are different. 

The above discussion suggest that, if we could construct a new 
equalizer such that the inputs to the gain controls are orthonormal, 
the eigenvalues of the correlation matrix of the new equalizer might 
be all equal and it might be possible to minimize the mean-square 
error in a single adjustment. Following this line of thinking, we obtain 
the generalized equalizer structure depicted in Fig. 3. As shown in 
Fig. 3, the equalizer input x(t) is connected to a bank of filters. The 
output of the ith filter is connected through a variable gain control d 
to the summing bus. The equalizer output is 

N 

y(t) = Ew(0, (54) 

i = l 

where z t (t) is the input to the ith gain control c,- . These signals 2,(0, 
i = 1 to N, are orthonormal when 

£, ZidTMIT) = 5,,- . (55) 

In the following, we show that, for any given channel, the filters in 
Fig. 3 can be designed to satisfy (55). We also show that, when (55) 
is satisfied, the mean-square error can be minimized by a single 
adjustment of the gain controls. 

Because of the press toward digitalization and integrated circuits, 
we shall specify the filters in Fig. 3 directly in digital filter form 
(instead of specifying them in analog filter form and then approxi- 
mating them with digital filters). We shall use nonrecursive digital 
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X(t) 




Fig. 3 — Generalized equalizer structure. 

filters 16 because they do not have stability problems and can be 
implemented most easily. 

A realization of the generalized equalizer using nonrecursive digital 
filters is depicted in Fig. 4. The N digital filters share the same tapped 
delay line. The ith digital filter, i = 1 to N, consists of the tapped 
delay line and the N coefficients P«, to P iN . Its output is 



*(0 = t,P*rtt - (3 - Dfli • = i t0 N - 



(56) 



The gain controls c x to C N are again adjusted to minimize the mean- 
square error e defined in (1). From (54) and (56), we can rewrite (1) 
into the following matrix form 



= c'PAP'c - 2c'Pv + £ dl 



(57) 




Fig. 4 — A new equalizer structure (a realization of the generalized equalizer 
structure using nonrecursive digital filters). 
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1987 



where c, A, and v have been defined previously [see (4), (5), and (7)]. 
The matrix P is defined hv 



P = 



PU P>2 



Pin 
P 2 ,v 



(58) 



-"jVl * .V2 ' ' * ,VA'J 

As in Section II, we define de/dc as the iV X 1 column vector whose 
ith element is de/dr,- . It can he shown from (57) that 



~ = 2PAP'c - 2Pv. 

dc 



(59) 



It can also be shown from (57) that c minimizes e if and only if 
de/dc = 0; therefore, from (59) the optimum value of c, c 0)lt , is 

c opt = (PT'A-'v. (60) 

We again define e as the difference between c and Co, lt ; hence, from 
(60) 

e = c - (PT'A-'v. (61) 

As in Section II, the initial values of c, e, and e are denoted, respec- 
tively, by c„ , e , and e,, . The values of c, e, and e after the fcth adjust- 
ment are denoted, respectively, by c k , e k - , and c* . From (61) 



e* = c, - (PT'A-'v, fc = 0, 1, 2, 
It can be shown from (57) and (62) that 



= Z dl - v'A~'v + elPAP'e 



k = 0, 1, 2, 



(62) 



(63) 



The fcth adjustment, k = 1, 2, 3, ••• , of the gain control is again 
made according to (15). From (62), (15), and (59), 



c* — c<t_i ; 



§*EL- 



(PO-'A-'v 



= c,_, - (P') _, A-'v - «,[PAP'c,_, - Pv]. 
From (62) , we have 

e t _, = c,_, - (PO-A-V. 
Using (65), we can rewrite (64) as 



(64) 
(65) 
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e k = e,_, - aJPAP'c,-, - Pv] 

= e*-, - a.PAP'tc,^ - (PT'A-v] 

= [I - a.PAP'je,-! , 

k = 1,2, ••• . (66) 

Now we can prove the first statement after (55) [that is, for any 
given channel, P can be chosen to satisfy (55) ] . It is easily shown 
from (58), (5), (6), and (56) that PAP' can be written in the form 



PAP' = 



6 2 1 622 ' ' ' "2,V 

-O.S'l 0;\'2 ' " ' 0,v.V_ 



(67) 



where 



&,, = E z<(!T)z,(!T). (68) 

! = -« 

Equation (55) is therefore equivalent to b u = 8 U , or 

PAP' = I. (69) 

From (18), 

A = QDQ'. (18) 

Let H be an N X N diagonal matrix whose ith. diagonal element is 
■\/\i , that is, 

VxT ••• 

VxT ••• 



H = 



(70) 



••• Vxjv, 

Then D = HIT and (18) becomes 

A = QHH'Q'. (71) 

Substituting (71) into (69) yields 

PQH(PQH)' - I. (72) 

Equation (72) holds if and only if PQH is an orthogonal matrix G. 
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From Q'Q = I, we can rewrite 

PQH = G (73) 

as 

P = GH -1 Q'- (74) 

For any given channel [i.e., for any given x(t)], we can compute a*,- 
from (6) and determine the eigenvalues Ai to A. v and the eigenvectors 
Ui to u.v . This determines D, Q, and H. There are an infinite number 
of N X A T orthogonal matrices. Any of them can be used as G (the 
choice of G will be discussed later) . Then from (74) P can be deter- 
mined, and this P satisfies (72), (69), and (55). This proves that, for 
any given channel, the digital filters in Fig. 4 can be designed to 
satisfy (55). 

Next we prove the second statement after (55). As in Section II, 
we use e„,in to denote the minimum value of e when c = c 01) , . Substitut- 
ing Copt in (60) into (57) yields 



« mi n = E £ ~ v'A-v. (75) 

Combining (63) and (75) gives 

e* = e min + e(.PAP'e. k - 0, 1, 2, ■ ■ ■ . (76) 

From (76) , the initial value of e is 

«o = e min + e.'PAP'c , (77) 

and the value of e after the first adjustment is 

ej = e„,in + e[PAP'e, . (78) 

From (66), e, = [I - a,PAP']e„ . Substituting this into (78), we have 

e> = e mJn + e,',[I - «,PAP']PAP'[I - a,PAP']e . (79) 

when P satisfies (74), (69) holds; i.e., PAP' = I. If we set «i = 1, 
I - ajPAP' = . Then from (79) ei = £„„„ . This proves that, when P 
satisfies (74) and «i = 1, the mean-square error e reduces to its 
minimum value c min after only one adjustment of the gain controls. 
In general it can be seen from (66) that, when P satisfies (74) and 
a k = 1, e A - = [I — atPAP'je;,-] = 0, regardless of the value of 
e /c _i . Consequently t k = emin , regardless of the value of < k -i . This 
means that each of the adjustments is capable of reducing the mean- 
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square error to its minimum value £ mlll , regardless of what is the value 
of c prior to that adjustment. 

Now we summarize this section. It is emphasized at the beginning of 
this section that it is the correlations between the inputs to the gain 
controls that determine the differences between the eigenvalues and 
consequently the rate of convergence of e. When the inputs to the 
gain controls are orthonormal, e can be minimized in one adjustment. 
The inputs to the gain controls can be made orthonormal by using the 
generalized equalizer structure depicted in Fig. 3. The generalized 
equalizer can be realized with digital or analog filters. A realization 
using nonrecursive digital filters is depicted in Fig. 4, and is analyzed 
in detail. It is shown that, for any given channel, one can design the 
digital filters from the simple equation (74) to orthonormalize the 
inputs to the gain controls. When the digital filters are so designed, 
each adjustment of the gain controls can reduce e to its minimum value, 
regardless of the value of e prior to that adjustment. Therefore, c 
can be minimized by only one adjustment of the gain controls. 

V. FURTHER PROPERTIES OF THE NEW EQUALIZER 

We have seen in the preceding section that it is possible to minimize 
e in one adjustment. In this section we show that such an improvement 
in convergence rate is obtained without changing the residue noise 
power, the minimum mean-square error, and the convexity of the 
adjustments, and without complicating the gain-control adjustment 
loop. 

5.1 Residual Noise Power and Minimum Mean-Square Error 

As described in Section II, during data transmission the equalizer 
output is sampled sequentially at time-instants t = ■ • • , U — T, t 2 , 
k + T, t 2 + 2T, • • • , and the time-samples are used to recover the 
transmitted information digits. Each of these time-samples consists of 
a signal and a noise component. The variance of this noise component 
(that is, the residue noise power) can be determined for both the new 
and the conventional equalizer. 

Consider first the new equalizer (Fig. 4) . Let the noise at the input 
of the equalizer be denoted by n{t). The resulting noise at the 
input of c ( is denoted by m(t), while the resulting noise at the 
equalizer output is denoted by v{t). Clearly 

m.(o - t,p<i<t - o' - on, * - 1 to N ( 8 °) 
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and 

v{t) = Zc,M,0). (81) 

We assume that the noise n(t) is a zero-mean stationary Gaussian 
process. Then m(t) ,i= 1 to N, and v(t) are also zero-mean stationary 
Gaussian processes. Consequently, the variance of v(t k ) is the same for 
all t k . It is therefore sufficient to consider a single t k (for example, 
t k = 0). From (80) and (81) 

"(0) - !>,. EW1 -j)T\. (82) 

i=i y=i 

Let n be an N X 1 vector whose ith element is n[(l — i)T], It can 
be easily shown from (82) that 

r(0) = c'Pn. (83) 

The noise vector n is distributed normally with zero mean. Let the 
co variance matrix of n be denoted by A. Then v(0) is distributed as 
the normal distribution with zero mean and variance c'PAP'c. Since c 
can be optimized in the training period, we assume that during data 
transmission c = c opt . Thus, during data transmission the variance 
of v(0) is Var [u(0)] - c£ pt PAP'c op , . Substituting (60) into above gives 

Var [v(0)] = v'A-'AA-'v. (84) 

Note from (84) that Var[v(0) ] is independent of P. 

Next consider the conventional equalizer. It can be easily seen that 
when P = I the new equalizer in Fig. 4 is reduced to the conventional 
equalizer in Fig. 1. Since Var[v(0)] in (84) remains unchanged when 
P is set to I, Var|>(0) ] of the new equalizer is equal to Var[y(0)] of 
the conventional equalizer. In other words, the use of the new equalizer 
does not change the residue noise power of the system. 

The minimum mean-square error, e,nin , has already been determined 
in the preceding sections. Comparing (12) with (75) shows that the 
minimum mean-square error does not change when the new equalizer 
is used instead of the conventional equalizer. 

5.2 Convexity of the Adjustment 

It has been shown 3,4 that, when the conventional equalizer is used, 
e is a strict convex function of the gain controls c. This ensures that e 
converges for all initial settings of c, and that e converges when the 
gradient method [equation (15)] is approximated by certain other 
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iterative techniques. In this subsection we show that t remains as a 
strict convex function of c when the new equalizer is used. 

Let 3C denote the open convex set jc: - « < c, < «, i - 1 to N). 
Let h be an N X 1 vector whose ith. element is h { . Clearly, 17 e is a 
function of class c (2) on the set OC. If we can show that 

t £ h > h < & > ° (85) 

for every c e JC and every h^O, then " e is strictly convex on 3C. 
It can be shown from (57) that 

= 26,, , all i and j (86) 



dc { dCj 

where b u has been specified in (68). From (86) and (67), we have 



N N d , e 



g E W, ^ = ^h'PAP'h. (87) 

It can be seen from (87) that (85) holds for every c t K and every 
h 9± if and only if PAP' is positive definite. Since P is a nonsingular 
matrix [see (74)], and A is positive definite, PAP' is positive definite. 
Therefore, (85) holds for every c t OC and every h^O, and consequently 
e is strictly convex on X. This proves that e remains as a strict convex 
function of c when the new equalizer is used. 

5.3 Gain-Control Adjustment Loop 

In the case of the conventional equalizer, the gain controls c are 
adjusted according to (15). The partial derivative dt/dci used in (15) 
is obtained 3 by correlating the time-samples of the tap signal and 
the time-samples of the error signal y(t) — d(t). 

In the case of the new equalizer, the gain controls c are again 
adjusted according to (15). Since 

N 

i/(0 = 2><s,(0, 

i = l 

I 1 = £ { t IV(W - d(kT)] 2 } 
oCi aCi U — oo ) 

= 2 fj [ Zi (kT)][y(kT) - d(m], 

i - 1 to N. (88) 
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It is seen from (88) that de/dci can be obtained similarly by correlat- 
ing the time-samples of z { {t) and the time-samples of y(t) — d(t). 
Therefore, the gain-control adjustment loop of the new equalizer is 
essentially the same as that of the conventional equalizer. 

VI. APPLICATION OF THE NEW EQUALIZER STRUCTURE 

In this and the next sections, we consider how to use the new 
equalizer structure for a Class IV partial-response system. It will be 
shown, both analytically and by computer simulation, that when the 
amplitude characteristic of the transmission medium does not vary 
appreciably from channel to channel (such as in the case of multiparty 
private line polling systems) , we can use a fixed P matrix for the new 
equalizer (that is, P i; - need not be adjusted in each training period). 
Such a simplification is possible even though the system timing, the 
demodulating carrier phase, and the phase characteristic of the trans- 
mission medium change from channel to channel. 

It has been shown in Section IV that c can be minimized in one 
adjustment if a k = 1 and P satisfies the equation 

PAP' - I. (69) 

As pointed out in Section 3.1, the elements a 1} of the A matrix are 
independent of the demodulating carrier phase, the system timing, the 
phase characteristics of the transmitting and receiving filters, and the 
phase characteristic of the transmission medium. Therefore, from (69) , 
P is independent of all these parameters. It can also be seen from 
(36) in Section 3.1 that a (j depends only on the amplitude charac- 
teristic |F 2 (/)| of the transmission medium (amplitude distortions of 
the transmitting and receiving filters are usually negligible). In some 
systems, such as private-line polling systems, \F 2 {j)\ does not vary 
appreciably from channel to channel. For such systems, a fixed P 
matrix may approximately satisfy (69) for all channels. Therefore, in 
designing such systems, we may estimate \F 2 (f)\ and compute from 
this an estimate of A (this estimate of A will be denoted by S). We 
can then estimate P from the equation 

PSP' = I. (89) 

and use this estimated P for all channels. When the estimated \F 2 (f)\ 
agrees with the actual \F 2 (f)\, S agrees with A. Consequently the 
estimated P satisfies (69), and e is minimized in one adjustment. When 
the estimated |F 2 (/)| differs from the actual \F 2 (f)\, the estimated P 
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will not satisfy (69) and the convergence rate of e will be reduced. 
In order to see whether the reduced convergence rate is satisfactory, 
we need formulas to relate the convergence rate to \F 2 (f)\. Such 
formulas will be derived in this section, and will be used in the next 
section for a hypothetical data communication system. 

Let the difference between the estimate of A and A be denoted by 
R, that is, 

R = S - A. (90) 

Since S is an correlation matrix, it is symmetric. Therefore, R is also 
a symmetric matrix. 

From (66) and (90) , we have 

e* = [I - a t PSP' + aaPRP'le*-! , k = 1, 2, • • • . (91) 
As in Section IV, we set 

a k = 1, all k. (92) 

Since the estimated P is computed from (89) , we can substitute (89) 
and (92) into (91 ) to obtain 

e, = PRP'e^ , k = 1, 2, • • ■ . (93) 

It can be easily seen from the recursive equation (93) that 

e* = (PRP')*e , k = 1, 2, •-• (94) 

From (76), the mean-square error after the kth adjustment is 

e* = 6 min + eJPAP'e* . (76) 

We now evaluate the last term e^PAP'e* in the equation above. It 
can be shown from (89) that 

P'P = S"\ (95) 

From (95), (94) can be rewritten as 

e k - PR(S- , R)*- I P'e„ , fc = 1, 2 ■ • • . (96) 

From (96) and (95), we obtain 

eJPAP'e, = e^P(RS- 1 ) A A(S- 1 R)*P'e , k = 1, 2, • • • . (97) 
Using A = S - R and (89) , we can rearrange (97) into the following 
form 

e[.PAP'e A = e5P(RS- 1 ) 2 *(I - RS^P^e,, , 

k = 1, 2, • • • . (98) 
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From (77), the initial value of mean-square error is 

eo = e min + efPAP'e,, • (77) 

It can be easily shown from (90) and (891 that 

e£PAP'e = e,',P[I - BS~ l ]p- l e . (99) 

Since A is symmetric, PAP' is symmetric. Let fr , fe , ' • ' , fa be the 
eigenvalues of PAP', and let Wi , w 2 , • ■ ■ w. v be a set of orthonormal 
eigenvectors of PAP' (w, corresponds to £,). From 

PAP'w, = f,w, (100) 

and [see (99)1 

PAP' = P[I - RS-'IP -1 (101) 

we have 

RS~'P _I w, = (1 - f.OP^'w, . (102) 

It is seen from (102) that the ith eigenvalue of RS -1 is (1 — f ,), and 
the ith eigenvector of RS"' is P _1 w, . We shall denote the ith eigenvalue 
ofRS" 1 byX.-so 

%i : = 1 - f , , i = 1 to N. (103) 

Now consider the matrix product P(RS -1 )"(I - RS~')P _1 in (98). 
To facilitate writing, we shall denote this product by 4», that is 

* = P(RS _1 ) 2 *(I - RS-')P" 1 . (104) 

From the eigenvalue and eigenvectors of RS -1 , we see that the ith 
eigenvalue of 4> is X'f (1 — X,), and the ith eigenvector of «I> is w, . 
We shall denote the ith eigenvalue of 4» by 5, , so 

5, = Xf (1 - X,). (105) 

We have shown in the above that the orthonormal eigenvectors 
w x to w,v of PAP' are also eigenvectors of 4>. Hence, PAP' and 4» can 
be simultaneously diagonalized by w, to w^ . By such diagonalizations 
we can reduce (99) to the form 

e^PAP'e,, = E (1 - X.-)(eSw,-) 2 (106) 

i-i 

and reduce (98) to the form 

e{.PAP'e, = Z M*(l ~ X,)(eJw,) 2 , k = 1, 2, • • ■ . (107) 
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It can be seen from (106) and (107) that the convergence rate of <■ 
depends on X t to \ N . Let | X, \ m „ denote the largest | X. | among | Xi | 
to | Xat |, that is, 

| a,- | , ^ | X, |, i = l to N. (108) 

Since PAP' is positive definite, its eigenvalues must all be positive. 
Hence, 

f , ■ = 1 - X. > 0, i - 1 to N. (109) 

From (106) to (109), the following bound is obtained 

e^PAP'e, g [ | X, | max ] 2 V,PAP / e , k = 1, 2, • • • . (110) 

Based on (110), we have the first method of estimating the con- 
vergence rate of e£PAP'e t . : 

First Method. From each | F 2 (J) I , compute X, to X* . This gives 
| X, | miix • According to (110), eJPAP'e t reduces by at least a factor of 
[ |X, | mux]"" after each adjustment. This completes the estimation. 

This method will be illustrated in the next section. In certain ap- 
plications, measurements of | F 2 (f) | may not be available. It may 
only be specified that | F 2 (/) | varies within certain bounds. In such 
cases, | X, | max cannot be determined. However, it can be bounded 
from the bounds of | F 2 (f) \ ■ It is shown in Appendix B that 

1 - h(F)U ^ X, ^ 1 - h(/)] rain , i = 1 to N. (Ill) 

where 

^~ Zi -'&'» • o *'*fc-'«- (u2) 



est 



As explained in Appendix B, \F 2 (f - / c )|act is the actual value of 
\F 2 (f - fc)\, \Fztf - /c)|est is the estimated value of \F 2 (f - f )\, 
[^(/)]mnx is the maximum value of r)(f) in the frequency range ^ 
f £ ft — fit and h(/)]min is the minimum value of v (f) in the 
frequency range oi ^ f = J2 — fi • 

Based on (110) to (112), we have the second method of estimating 
the convergence rate of e£PAP'e* : 

Second Method. From the bounds of | F 2 (f) \ , determine fo (/)]«,.« 
and b(/)] min from (112). If 

I 1 - [u(/)ki- I > I 1 - h(/)l»« L 
use | 1 — [r?(/)] min I as the upper bound of | X, | m „ . If 

1 1 - fo(/)U ! > 1 1 - fo(/)U I , 
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use | 1 — h(/)] ran x | as the upper bound of | X, \ max . Use the upper 
bound of | X, | max as | X, | m , lx to obtain from (110) an estimate of the 
minimum convergence rate of e^.PAP'ei . 

As illustrated in the next section, the second method requires only 
some simple hand-calculations, and hence may be applied first even 
if measurements of | F 2 {j) | are available. However, it should be warned 
that the second method is usually too pessimistic because the upper 
bound of | X, | innx , obtained in the above fashion, can be several times 
larger than the actual | X, | B „ . (Consequently, the convergence rate 
may appear to be unsatisfactory when it is actually satisfactory.) 
Therefore, for borderline cases, the first method should be used. 

We illustrate these two methods in the next section. 

VII. NUMERICAL EXAMPLES AND COMPUTER SIMULATION 

In order to illustrate the methods we consider a hypothetical private- 
line data communication system (hereafter referred to as the 
hypothetical system). We assume that the system uses a Class IV 
partial-response signal and transmits over voiceband at a baud rate 
of 4800 bauds. The cutoff frequencies f t and / 2 defined in Section 3.1 
will be 400 and 2800 Hz respectively. The demodulating carrier fre- 
quency f c is equal to / 2 . It is assumed that the new equalizer in Fig. 4 
is used with N = 13 and a prefixed P matrix computed from (89) 
(see footnote in Section 7.1). The two methods will be used in Sections 
7.1 and 7.2, which follow, to estimate the convergence rate of e^AP^. 
! the convergence rate of the mean-square error e is identical to the 
convergence rate of e' k PAP'e k [see (76) in Section IV]). A third and 
very elaborate method, computer simulation of the data communication 
system, will be used in Section 7.3 to demonstrate the convergence 
rate. 

7.1 Estimation by the First Method 

Since | F 2 (f) | of private lines does not deviate considerably from 
a constant in the frequency band 400 to 2800 Hz, we shall simply use 
a constant as the estimate of | F 2 (f) | in this frequency band, and 
compute a P matrix accordingly from (89). This prefixed P matrix 
will be used for all lines. 

Now consider what happens to the convergence rate when the private 
line has a | F 2 (f) \ as depicted in Fig. 5. It can be seen that this | F 2 (f) \ 
varies from —4.8 dB to 0.4 dB, and then to —6.3 dB in the frequency 
band 400 to 2800 Hz. Such a variation is large for private lines. How- 
ever, we show that even for this large variation the prefixed P matrix 
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Fig. 5— The \F,(f)\ used in Section 7.1. 

can be used and the convergence rate of the mean-square error is 
satisfactory. 

In order to use (110), we compute the eigenvalues % l to X w in the 
following steps: 

Step 1. Compute the elements a,, of the A matrix from (36) and 
the | F 2 (f) | in Fig. 5. It is sufficient to compute only the first row of A. 
To see this, note from (6) that a a satisfies the condition 

a„ = a„ (113) 

and the condition 

a„- = «<,+*><*+*> • (H4) 

From (113) and (114), we see that A can be written in the form 



A = 



a,, 


«12 


Gl3 


a, 4 


d\N 


«12 


On 


a, 2 


da 


• • dnif-i) 


a ia 


dy. 


a,, 


dl2 


&HAT-2) 


On 


ffia 


«12 


a n 


a 1( Ar-3) 



(115) 



_ai,v ttl(,V-l) fll(Ar-2) fll(.V-3) '*' a ll 

where the elements on the diagonal line are all equal to an , and the 
elements on each off-diagonal line are equal (for example, the ele- 
ments on the second off-diagonal line are all equal to a 13 ) . Thus, it is 
sufficient to compute only the first row of A. 
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Step 2. Compute the S matrix. It can be seen from Section 3.1 
(from (37) to (40)) that, when 



we have 



where 



F 2 {j) | = a constant, j x ^ / ^ / 2 , 
A - OnA, 



(116) 
(117) 



A = 



1 O-i ••■ 

1 -i ••• 

§ 1 ••• 

-\ 1 ••• 



(118) 



■•• 1. 
Since we have used a constant as the estimate of |F 2 (/) |, we have 

S = a„A. (119)* 

Step 3. Compute R = S — A, and compute the eigenvalues ^ 
to A* of RS" 1 . 

The computations in the above three steps can best be carried out 
by a computer program. Such a program has been written. For the 
| F 2 (j) | in Fig. 5, the eigenvalues of RS -1 are found to be: —0.1573, 
-0.1434, -0.0902, -0.0615, -0.0034, 0.0242, 0.0716, 0.1084, 0.1834, 
0.1945, 0.2954, 0.3797, and 0.4651. The largest magnitude of these 
eigenvalues is 0.4651; therefore, 

I A, Lx = 0.4651. (120) 

Substituting (120) into (110) gives 

e^PAP'e, ^ Ij^J e^PAP'e,, . (121) 

It can be seen from (121) that e*PAP'e t reduces at least 4.62 times 



*From (119) and (89), we may determine the P matrix. Since P is not needed 
for the following computation, we do not carry out such calculations here. How- 
ever, it should be pointed out that the method from (69) to (74) may be used 
for such calculations. It should also be noted that by setting the G matrix in 
(74) to I, we can reduce most of the elements in P to zero. For example, when 
N is & multiple of four, it is sufficient to implement only N 2 /S of the P t j s in 
Fig. 4. This greatly simplifies the implementation. 
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after each adjustment. In other words, each adjustment reduces the 
mean-square error by at least 6.65 dB. This convergence rate is satis- 
factory. For example, let e min be 0.01 and let the initial mean-square 
error e be as large as 4 (see computer simulation in Section 7.3). From 
(76), (77), and (121), we see that after four adjustments (approximately 
16 milliseconds) the mean-square error e reduces to less than 0.0187. 
Clearly, this convergence rate is satisfactory. 

7.2 Estimation by the Second Method 

When measurements of | F 2 {F) | are not available, we may estimate 
the convergence rate from the bounds of | F 2 (f) | . For illustrative 
purposes, let us assume that | F 2 (f) | does not deviate from unity by 
more than -2.5 dB or 1.5 dB, that is, 

-2.5 g 20 log 10 | F 2 (f) | ^ 1.5, 400 ^ / ^ 2800. 

Notice that | F 2 (f) | can vary in any manner within these bounds. 
We simply use unity as the estimate of | F 2 (f) \ . Then from (112), 
h(/)U = 1-414 and [r,(/)] min = 0.562. Since | 1 - h(/)] min | > 
I 1 - foil)]™* I, we use | 1 - h(/)] min I as the upper bound of | X, | max , 
that is, | X, U, ^ 0.438. Using | X,- | m « = 0.438, we obtain from (110) 



* [hh 



elPAP'e, ^ I f^j eSPAP'e,, . 

Thus, ejPAP'e k reduces at least 5.2 times after each adjustment (a 7.16 
dB reduction per adjustment). Notice that since the actual value of 
I X, \ max may be much less than 0.438, the actual convergence rate can 
be much faster. 

7.3 Computer Simulation of the Hypothetical System 

The analytical methods in the preceding subsections yield bounds 
of the convergence rate. In order to demonstrate the actual con- 
vergence rate, we simulate the hypothetical system on the computer, 
using a number of private-line characteristics obtained from field 
measurement. In order to conserve space we describe here only the most 
important results. Fig. 6 shows the convergence of the mean-square 
error for a private line which has severe amplitude and phase dis- 
tortions (see Table I). Other lines simulated have less severe distor- 
tions and hence faster convergence rates. A 30-dB signal-to-noise 
ratio is assumed at the equalizer input. The bottom curve shows the 
convergence of e when the new equalizer is used (with a prefixed P 
matrix computed from (119) and (89)). It can be seen that, when the 
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Fig. 6 — Convergence of the mean-square error. 



Table I — Relative Envelope Delay and Relative Loss Charac- 
teristics of Private Line used in Section 7.3. 
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new equalizer is used, e converges rapidly to its minimum value, and 
the training period may be terminated after the third adjustment 
(three adjustments require approximately 12 milliseconds). These 
simulation results are in full agreement with the theoretical results 
in the preceding subsections. 

The top curve in Fig. 6 shows the convergence of e when the con- 
ventional equalizer in Fig. 1 is used (also with N = 13). It can be 
seen that the convergence of e is rather slow, and that it takes more 
than 11 adjustments (approximately 44 milliseconds) to reduce t to 
close to its minimum value. 

Figure 6 is obtained with a specific setting of system timing and 
demodulating carrier phase. The computer program has also been 
executed for a sufficiently large number of other timing and carrier 
phase settings. Curves similar to those in Fig. 6 have been obtained. 
The results may be summarized in Fig. 7. The horizontal axis in Fig. 7 
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Fig. 7 — Number of equalizer adjustments required to reduce the mean-square 
error to less than 0.05 for different timing and carrier-phase settings. 
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shows the setting of the timing and the carrier phase, while the 
vertical axis shows the number of adjustments required to reduce 
the mean-square error e to less than 0.05. For example, when timing 
is set to 0.1T and the carrier phase is set to 90 degrees, three adjust- 
ments arc required to reduce e to less than 0.05 when the new equalizer 
is used, while 11 adjustments are required when the conventional 
equalizer is used. It can be seen from the top curve in Fig. 7 that when 
the conventional equalizer is used, the number of required adjustments 
varies a great deal with timing and carrier phase, and as many as 19 
adjustments can be required. On the other hand, when the new equalizer 
is used (bottom curve), the number of required adjustments remains 
small for all time and carrier-phase settings. (This is due to the first 
convergence of the new equalizer.) Thus, when the new equalizer is used, 
it is not necessary to make fine adjustments of timing and carrier 
phase at the beginning of the training period. We may simply set these 
parameters to some reasonable values, and then use the new equalizer 
to quickly reduce the mean-square error. It is possible that the whole 
process of adjusting the timing, the carrier phase, and the equalizer 
can be completed in a brief training period. 

VIII. SUMMARY AND CONCLUSIONS 

Section II considers amplitude modulation data communication sys- 
tems that use transversal filters for automatic equalization. The gain 
controls of the transversal filters are adjusted by the gradient method 
[equation (15)] to minimize the mean-square error between the 
received and the desired pulses. After the fundamentals are reviewed, 
the mean-square error is decomposed into A' error-components (N is 
the number of taps on the transversal filter). The error-components 
depend on the eigenvalues Ai , • • • , A.v of the correlation matrix A. 
These eigenvalues and the convergence of the error-components 
depend on the signaling scheme and the channel characteristics. Two 
important classes of digital communication systems are examined: the 
class IV partial-response system and the SSB Nyquist system. It is 
shown that for both systems, the engenvalues A x , • • • , A.v are inde- 
pendent of the demodulation carrier phase, the system timing, the 
phase characteristic of the transmission medium, and the phase 
characteristics of the transmitting and receiving filters. Consequently, 
the eigenvalues depend on only the amplitude characteristics of the 
transmission medium and the filters. Since amplitude characteristics of 
the filters depend on the signaling scheme, it might be said that the 
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eigenvalues depend on only the signaling scheme and the amplitude 
characteristic \F 2 (f)\ of the transmission medium. For applications 
where \F 2 (f)\ does not vary considerably from channel to channel 
(such as private-line systems), the eigenvalues depend primarily on 
the signaling scheme. For the SSB Nyquist system, the eigenvalues 
Ai , • • • , Ay are nearly equal. Consequently, as shown in Section 3.2, 
each adjustment reduces each of the error-components by a large 
factor, and the mean-square error can be minimized after only a few 
adjustments (for example, two adjustments). For the class IV partial- 
response system, the eigenvalues Ai , • • • , A. v are very different in 
magnitude. (The ratio of the maximum eigenvalue to the minimum 
eigenvalue is very large; e.g., 40 or larger.) Therefore, each adjustment 
cannot reduce each error-component by a large factor, and, as can be 
seen from Figs. 6 and 7, the convergence rate may not be satisfactory 
for fast start-up purposes. 

The results in Section III suggest that in order to improve the con- 
vergence rate an attempt should be made to reduce the differences 
between the eignevalues. As emphasized in Section IV, differences in 
eigenvalues are caused by the correlation between the inputs to the 
gain controls. When the inputs to the gain controls are orthonormal, 
the eigenvalues are all equal and the mean-square error can be min- 
imized in only one adjustment. The inputs to the gain controls can be 
made orthonormal by using the generalized equalizer structure in Fig. 
3. The generalized equalizer can be realized with digital or analog 
filters. Because of the press toward digitalization and integrated cir- 
cuits, digital filters are used. A realization using nonrecursive digital 
filters is depicted in Fig. 4. This equalizer structure, referred to as 
the new equalizer structure, is analyzed in detail. It is shown that for 
any given channel the coefficients of the digital filters can be set 
according to (691 or (74) to make the inputs to the gain controls 
orthonormal. Then each adjustment of the gain controls can reduce 
the mean-square error e to its minimum value, regardless of the mag- 
nitude of e prior to that adjustment. In Section V it is shown that such 
a fast convergence (one-step convergence) is obtained without chang- 
ing the residual noise power, the minimum mean-square error, and the 
convexity of the adjustments, and without complicating the gain- 
control adjustment loops. 

The theory in Sections IV and V is general in that it applies regard- 
less of the type of modulation (SSB, VSB, or DSB) or the signaling 
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scheme (partial-response or Nyquist). Sections VI and VII consider 
the application of the theory to a Class IV partial-response system. 
Most importantly, it is shown that the coefficients P i7 of the digital 
filters in the new equalizer can be prefixed. It is first observed that 
for Class IV partial-response systems the coefficients P„ in the new 
equalizer depend only on the amplitude characteristic |P>(/)| of the 
transmission medium. Consequently, for private-line systems and sys- 
tems where \F»(f) | does not vary considerably from channel to channel 
(delay distortion can vary arbitrarily), we may compute the co- 
efficients Pjj from an estimated (typical) \F-Aj)\ and use these pre- 
fixed Pjj for all channels. One-step convergence is obtained when actual 
|^a(/)| agrees with the estimated |F 2 (/)|. When actual \F 2 (/) | differs 
from the estimated ^(/Jli the convergence rate of the mean-square 
error is reduced. In order to determine if the reduced convergence rate 
is satisfactory, two analytical methods are developed to relate the 
convergence rate to |F2(/)|. These methods are first presented in 
Section VI, and are used in Section VII for a hypothetical data com- 
munication system. This hypothetical system uses a Class IV partial- 
response signal and transmits over private voice lines at a data rate 
of 4800 bauds. For private lines, \F.,(f) | does not vary considerably in 
the passband 400 to 2800 Hz; therefore, a constant is used as 
the estimate of |F L >(/.)| and the prefixed P, ; - are computed from 
(89) and (119). It is shown analytically in Section 7.1 that the 
use of the prefixed P,-; yields a satisfactory convergence rate even 
when the line has severe amplitude distortion. To further demon- 
strate the convergence rate, the hypothetical system is simulated on a 
digital computer, using the prefixed P,; and a number of private-line 
characteristics obtained from field measurements. A 30-dB signal-to- 
noise ratio at the equalizer input is assumed. Figs. 6 and 7 illustrate 
the convergence of the mean-square error for a private line that has 
severe amplitude and phase distortion. (The convergence is faster 
for other lines.) It can be seen from these figures that, when the new 
equalizer is used, the equalizer settles after three adjustments (ap- 
proximately 12 milliseconds I . Furthermore, because of the fast con- 
vergence, the settling time remains small for all settings of timing and 
demodulating carrier phase. Thus, when the new equalizer is used, it is 
not recessary to make fine adjustments of timing and demodulating 
carrier phase in the start-up period. These parameters can simply be 
set to some reasonable values at the beginning of the start-up period. 
This can further reduce the overall start-up time of the system. 
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APPENDIX A 

A.l Determination of Eigenvalues and Eigenvectors 

In this appendix we determine the eigenvalues and eigenvectors of 
the N X N matrix A in equation (40). The two cases of odd N and 
even N are considered separately. 

Case 1. Odd N. 

Let A be an eigenvalue of A, and let 

Wi 



u = 



Ma 



Un. 



(122) 



be the corresponding eigenvector. We can rewrite the equation A u = 
A u as 

N 

X) a,-,u, = \Ui , i = l to AT. (123) 

Equation (123) can be split into the following two equations: 

N 

X) a./W, = \u { , i = odd (124) 



/ = i 



and 



X a fi Uj = \m, , i = even. (125) 

; = 1 

From (39), a., v = when i — j is odd. Thus (124) can be reduced to 
y^ dijUj = Xm, , * = 1, 3, ■ • • , N. 

1-1,8, • • • ,.V 

Using (39) , we can write the above equation as 
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(126) 



0- 

The first matrix on the left-hand side of (126) is an (N + l)/2 X 
(N + l)/2 tridiagonal matrix. In addition to the trivial solution 

Wi = u 3 = u 5 = • • • = u N = 0, 

there are (N + l)/2 nonzero solutions to (126). The kth such solu- 
tion, k = 1,2, •■■ , (N+ l)/2,is 14 

kir 



X = 1 — cos 



N + 1 



(127) 



and 



N + 1 



2 



+ 1 



sm 



sin 



+ 1 

k-rr 



N + 1 



2 



+ 1 



k2r 



N + 1 



+ 1 



sm 



, N_±1 

K ~ IT 



N + 1 



+ 1 



(128) 



From (39), (125) can be written as 
• •• 

! -| ... 



1 ~h 







1 u 2 




u 2 


111 




Ui 


u 6 


= X 


We 


LU(K-l)_ 




_M(AT-i)_ 



(129) 
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where the first matrix on the left-hand side is an (N — l)/2 X 

(N _ i)/2 tridiagonal matrix. In addition to the trivial solution 

u 2 = u t = u 6 = • • • = W(Ar-n = 0, 

there are (N - l)/2 nonzero solutions to (129). The fcth such solu- 
tion, k = 1,2, ••• , (N- l)/2,is 



A = 1 — cos 



k-rr 



N - 1 



+ 1 



(130) 



and 



M, 



u, 



W(AT-l) 



N - 1 



+ 1 



sin 



sin 



kir 



N - 1 



+ 1 



k2w 



N - 1 



+ 1 



k — x — r 



sin 



N - 1 



+ 1 



(131) 



It can be shown that the A given by (127) is not equal to that given 
by (130). Therefore, (126) and (129) do not have the same nonzero 
solution. 

We have split (123) into (124) and (125), and rewritten (124) and 
(125), respectively, as (126) and (129). Thus, A and u satisfy (123) 
if and only if they satisfy both (126) and (129). Since (126) and 
(129) do not have the same nonzero solution, A and u are given either 
by the nonzero solution of (126) plus the trivial solution of (129), or 
by the nonzero solution of (129) plus the trivial solution of (126). 

The above may be summarized in the form of a lemma: 

Lemma A-l: For odd N, the eigenvalues and eigenvectors of A may be 
divided into two groups. The kth eigenvalue and eigenvector, k = 1, 2, • • ■ , 
(N + l)/2, in the first group are given by (127), (128), and 



u 2 = u t = u 6 = 



= u<tf-n = 0. 
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The kth eigenvalue and eigenvector, k = 1, 2, • • • , (N — I)/ 2, in the 
second group are given by (130), (131), and 

Ui = u 3 = u B = ••• = u N = 0. 

It can be shown that the eigenvectors specified in Lemma A-l form 
a set of N orthonormal eigenvectors. 

Case 2. Even N 

Using the same method as in Case 1, one can verify the following 
lemma: 



Lemma A-2: For even N, the eigenvalues and eigenvectors of A may 
be divided into two groups. The kth eigenvalue and eigenvector in the first 
group, k = 1, 2, • • • , N/2, are given by 
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A - 7r 
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(132) 
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u 2 = u A = u 6 = • ■ ■ = u N = 0. 



(133) 



(134) 



The kth eigenvalue and eigenvector in the second group, k = 1,2, • • • , N/2, 
are given by 



X = 1 — cos 



h 



2 ^ l 



(135) 
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and 



u 2 






kr 
sm 

f +1 


Ui 
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r 2 v 


. k2ir 
sin 

f +1 


.!- 


U N 




fc — 7T 

sin^ 

2 +1 . 



(136) 



and 



W, = W 3 = U 5 = 



= Uijf-t) = 0. 



(137) 



The eigenvectors given by (133) and (134) together with the 
eigenvectors given by (136) and (137) form a set of N orthonormal 
eigenvectors. 

APPENDIX B 

B.l Bounds on Eigenvalues 

It has been defined in Section VI that X< , i = 1 to N, are the eigen- 
values of RS~\ In this appendix we study the relation between X< 
and the amplitude characteristic | F 2 (f) \ of the transmission medium. 
It is easier to first study the relation between | F 2 (f) \ and the eigen- 
values of AS -1 . Let X denote the eigenvalue of AS -1 . Since AS -1 and 
S~'A have the same eigenvalues, X is also the eigenvalue of S _I A. Thus 

S _, Ax = ax, (138) 

where x is the eigenvector of S _1 A. Premultiplying both sides of (138) 
with X'S gives 

x'Ax = ax'Sx. (139) 

Since S is a correlation matrix like A, S is positive definite. Consequently 
x'Sx > for every x except £ = 0. Since x is an eigenvector, x ^ 0. 
Therefore 



x'Sx > 0. 
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From (139), we have 

X = ££. (140) 

In this appendix, c is the vector denned in (4) for the conventional 
equalizer. Let [(c'Ac)/(c'Sc)] m i n denote the minimum value of (c'Ac)/ 
(c'Sc) over all c ^0, and let [(c'Ac)/(c'Sc)] mnJ[ denote the maximum 
value of (c'Ac)/ (c'Sc) over all c 5^ 0. Then since the eigenvectors X 
are elements of the set (c | c 7^ 0), we have 

l~c'Acl .xAI' Tc'Acl , . 

L?5L. = Fsi = ys^_L • (141) 

Now we evaluate the two bounds in (141). Using the notations in 
Section II, we can easily show that 

c'Ac = £ y 2 . (142) 

Since y L are time samples of y(t) taken at the Nyquist rate, we have 
by sampling theorem 

£ y\ = 2(f. - /■) P [y(t)f dt. (143) 

According to Parseval's theorem 

/_" W)f dt = /_" Y(f)Y*(f) df, (144) 

where Y(f) is the Fourier transform of y(t), and Y* (j) is the com- 
plcx conjugate of Y(f). From (2), we obtain 

}-(/) =X(J)e J2 " T Zcfi-' 2 "". (145) 

*=i 

Substituting (145) into (144) gives 

P W)Y di = P \X(f)\* \tc# J2t ' kT ]\Zc# J * w ' kT ] df. (146) 

J-00 •'-00 La = i _ _*=i J 

One can rearrange (146) into the following form 

P [y(t)f dt = 2 P |Z(/)| 2 r/(/) d/, (147) 

w - 00 « 

where 
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?(/) = [Z o k cos 2rfkTj + [g c.sin 27r/fc7 1 ] 2 . (148) 

Since we are considering a Class IV partial-response system, we can 
use (30) in Section 3.1 to determine \X(f)\, and use (35) for the 
product \FAf - fc)Fs(f ~ fc) | in \X(f)\. From these equations, (147) 
can be reduced to the following form 

/" \y(t)] a dt = \ £'" ' \F 2 (f - j e )\ 2 [sin jfzjj <?(/) df. (149) 
Combining (142), (143), and (149) gives 

c'Ac = {U - /,) / o '' " \F 2 (f - f c )\ 2 [sin jfrjj Q(1) df. (150) 

In the following discussion, we have to distinguish between the actual 
1*2 (/ - /r)| (denoted \F 2 (f - /r)|act) and the estimated \F,(f - /,.) | 
(denoted \F->(f — /«,) | eB t). In order to emphasize that the \F 2 (f — fc) \ 
in (150) is the actual \F,(f - /,) |, we rewrite (150) as 

c'Ac = (f 2 - /,) j'j " [\F 2 (f - / f )| act ] 2 [sin jfrjj 90) df. (151) 

When the actual \F 2 (f — /,.) | is replaced by the estimated \F 2 (f — /<•) |, 
A is replaced by S, and (151) becomes 

c'Sc = (f 2 - /,) /"' " [\F 2 (1 - j c )\ eat f sin^y ' <?(/) df. (152) 
Let 7j {f) be defined as 

" (/) = IK = l:fc r **tsu-t.. (153) 

From (151), (152), and (153), we obtain 

c'Ac f " " ( > )Q( fl <" 



c'Sc 



r " "(/) df 

•'O 



(154) 



where 



m = [\Uf - / e )|..t]'[sin J^l\ 2 -q(D- (155) 

It can be easily seen from (155) and (148) that 

0(/) £ 0, 0g/g/ 2 -/,. (156) 
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Furthermore, for c^O, nf/l cannot be identically in the frequency 
range ^ / ^ / 2 — /i (for if so c'Sc would be 0, contradicting the 
fact that S is positive definite). It can be seen from (153) that 



v u) *> o, ogfgn-u. 

From these properties of fi(/) and ij(f), one can see that 



r 



10)00) dj 



f 



^ fo0)U , 



(157) 



(158) 



m ^ 



where [rj(f) ] mnx is the maximum value of 7}{f) in the frequency range 
^ / ^ / 2 — /i . It can alse be seen that 



I 



/,-/i 



viJMDdf 



f 



^ W/)] min , 



(159) 



m dj 



where [•>?(/) ] min is the minimum value of -q{j) in the frequency range 
g / g / 2 - / t . Now we tie the results together. From (140), (141), 
(154), and (158), we have 



X " x'Sx - 



c'Ac 
.c'Sc. 



^ fo0)l, 



Similarly, from (140), (141), (154), and (159), 

"c'Ac 



X x'Sx - 



c'Sc 



^ h(f)U 



We may combine (160) and (161) as 

[„(/)] min ^ X ^ [,(/)],„ . 



(160) 



(161) 



(162) 



It can be easily shown that the eigenvalue X, of RS ' is related to the 
eigenvalue X of AS -1 by 

1 - X, = X. (163) 

From (163) and (162), we obtain the final result 

1 - fo(/)U ^ X, ^ 1 - [,(f)U , t = 1 to N. (164) 

Equations (164) and (153) are, respectively, (111) and (112) in 
Section VI. 
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